Image recognition apparatus

ABSTRACT

An image recognition part of an image recognition apparatus recognizes an object based on a target area in an outside-vehicle image obtained by a camera installed in a vehicle. A position identifying part identifies an optical axis position of the camera relative to the vehicle based on the outside-vehicle image, and an area changing part changes a position of the target area in the outside-vehicle image according to the optical axis position of the camera. Therefore, it is possible to recognize an object properly based on the target area in the outside-vehicle image even though the optical axis position of the camera is displaced.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates to technologies for recognizing an object based onan image.

2. Description of the Background Art

Recently, an image recognition apparatus for recognizing an object basedon an outside-vehicle image, which is obtained by a camera installed ina vehicle and shows a periphery of the vehicle, is proposed. This imagerecognition apparatus is considered to be used in a variety ofapplications in a vehicle.

The image recognition apparatus, for example, recognizes an object basedon an outside-vehicle image and in a case where there is an objecthaving a possibility of collision, the image recognition apparatus warnsa user of the possibility of collision. Thereby, the user can drive inconsideration of the object. In addition, the image recognitionapparatus recognizes an object based on an outside-vehicle image andtransmits a signal that indicates existence of the recognized object toa vehicle controller for controlling a behavior of the vehicle. Thereby,the vehicle controller can control the behavior of the vehicle inconsideration of the object.

To fulfill a recognition function of this image recognition apparatusproperly, an optical axis of a camera is needed to point in apredetermined direction relative to the vehicle. However, the opticalaxis of the camera, which is installed so as to point in thepredetermined direction, may be displaced with age. For this reason, theimage recognition apparatus for informing a user that the optical axisof the camera is changed largely from the predetermined direction in acase where the optical axis of the camera is changed largely from thepredetermined direction is proposed. Thereby, the user can adjust theoptical axis of the camera to the predetermined direction by suchinformation being informed.

However, it is quite a trouble for a user to adjust an optical axis ofthe camera every time the image recognition apparatus detectsdisplacement of the optical axis position. Therefore, improvements wererequired.

SUMMARY OF THE INVENTION

According to one aspect of this invention, an image recognitionapparatus that recognizes an object based on an image, includes: arecognition unit that recognizes the object based on a target area in animage obtained by a camera installed in a vehicle; an identifying unitthat identifies an optical axis position of the camera relative to thevehicle based on the image; and a changing unit that changes a positionof the target area in the image according to the optical axis positionidentified by the identifying unit.

The position of the target area in the image is changed according to theoptical axis position of the camera, and therefore, the object can berecognized properly based on the target area even though the opticalaxis position of the camera is displaced.

According to another aspect of this invention, the image recognitionapparatus further includes: a derivation unit that derives an amount ofdisplacement of the optical axis position from an initial position basedon the image; and an informing unit that informs a user that the amountof displacement is more than a threshold value in a case where theamount of displacement is more than a threshold value.

In the case where the amount of displacement of the optical axisposition of the camera from the initial position is more than thethreshold value, a user is informed of the displacement. Thereby, theuser can recognize that the optical axis position of the camera isdisplaced largely.

According to another aspect of this invention, the image recognitionapparatus further includes: a derivation unit that derives an amount ofdisplacement of the optical axis position from an initial position basedon the image; and an adjustment unit that adjusts the optical axisposition in a case where the amount of displacement is more than athreshold value.

In the case where the amount of displacement of the optical axisposition of the camera from the initial position is more than thethreshold value, the optical axis position is adjusted. Thereby, a usercan skip a step to adjust the optical axis position of the camera eventhough the optical axis position of the camera is displaced largely.

Therefore, an object of the invention is to recognize an object properlybased on a target area even though an optical axis position of a camerais displaced.

These and other objects, features, aspects and advantages of theinvention will become more apparent from the following detaileddescription of the invention when taken in conjunction with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an appearance of an image recognition apparatus;

FIG. 2 shows condition in which a camera is installed in a vehicle;

FIG. 3 shows a direction in which an optical axis of a camera ispointed;

FIG. 4 mainly shows a configuration of a body part of a firstembodiment;

FIG. 5 shows an exemplary outside-vehicle image;

FIG. 6 is a diagram to explain a method to derive a vanishing pointposition;

FIG. 7 shows a change of a vanishing point position;

FIG. 8 shows a flow of a collision-warning process;

FIG. 9 shows a flow of an optical-axis-related process in a firstembodiment;

FIG. 10 shows a configuration of a camera in a second embodiment;

FIG. 11 mainly shows a configuration of a body part in a secondembodiment;

FIG. 12 shows a flow of an optical-axis-related process in a secondembodiment;

FIG. 13 shows an exemplary outside-vehicle image;

FIG. 14 shows an exemplary outside-vehicle image;

FIG. 15 shows an exemplary outside-vehicle image; and

FIG. 16 shows an exemplary outside-vehicle image.

DESCRIPTION OF THE EMBODIMENTS

Hereinbelow, embodiments of this invention are explained with referenceto the attached drawings.

<1. First Embodiment>

FIG. 1 shows an appearance of an image recognition apparatus 2 in thefirst embodiment. This image recognition apparatus 2 is installed in avehicle (hereinafter, referred to as an “own vehicle”) and mainlyincludes a body part 20 and a camera 3. The body part 20 and the camera3 are electrically connected through a cable 19, and they cantransmit/receive a signal mutually. The camera 3 captures a surroundingarea of an own vehicle and obtains an outside-vehicle image that showssurroundings of the own vehicle. The body part 20 recognizes anothervehicle running around the own vehicle based on a target area in anoutside-vehicle image obtained by the camera 3. In addition, in a casewhere the body part 20 recognizes another vehicle and there is a highpossibility of collision between the own vehicle and another vehicle,the body part 20 warns a user of the high possibility of collisionbetween the own vehicle and another vehicle. Furthermore, the body part20 identifies an optical axis position of the camera 3 and changes aposition of the target area in an outside-vehicle image according to theoptical axis position. Hereinbelow, a configuration and processing ofthe image recognition apparatus 2 having such functions are explained indetail.

<1-1. Configuration of Camera>

First, a configuration of the camera 3 is explained. As shown in FIG. 1,the camera 3 includes an optical lens 10, a chassis 11 and a holdingpart 12. The optical lens 10 is an optical device that focuses light. Anangle of view of the optical lens 10 is, for example, 45 degrees. Thechassis 11 is a chassis that stores parts such as the optical lens 10and an imaging device (illustration is omitted). The holding part 12 isa holding member that holds the chassis 11. The chassis 11 is fixed tothe holding part 12 with a screw 14. It is possible to adjust an angleof the chassis 11 relative to the holding part 12 by the screw 14 beingloosened. The holding part 12 includes a mounting surface 13 touchingwith an inner surface of a windshield of an own vehicle by planecontact.

FIG. 2 shows condition in which the camera 3 is installed in an ownvehicle 1. The camera 3 is installed in a cabin of the own vehicle 1 bythe mounting surface 13 of the holding part 12 being touched with aninner surface of a windshield 15 by plane contact. The mounting surface13 has a double-sided tape and the mounting surface 13 is fixed to thewindshield 15 with this double-sided tape. Thereby, the holding part 12is adhered near a rearview mirror 16 installed on the inner surface ofthe windshield 15.

In this way, the chassis 11 is fixed relative to the own vehicle 1 bythe holding part 12 being adhered in the inner surface of the windshield15. In other words, an optical axis of the camera 3 (an optical axis ofthe optical lens 10) is pointed in a predetermined direction withreference to the own vehicle 1. Thereby, as shown in FIG. 3, an opticalaxis of the camera 3 is pointed to the front of the own vehicle 1.Therefore, the camera 3 captures a forward area of the own vehicle 1 andobtains an outside-vehicle image showing the forward area of the ownvehicle 1.

However, the chassis 11 may move relative to the holding part 12 due toloosening of the screw 14 caused by aging and the like. In this way, ifthe chassis 11 moves relative to the holding part 12, an optical axisposition of the camera 3 relative to the own vehicle 1 is displaced froma predetermined direction. Normally, the optical axis of the camera 3 isdisplaced downward relative to the own vehicle 1 with age.

<1-2. Configuration of Body Part>

Next, a configuration of the body part 20 is explained. FIG. 4 mainlyshows the configuration of the body part 20. The body part 20 includes arectangular solid chassis and this chassis stores a variety ofelectronic parts. The body part 20 mainly includes an image processingpart 21, a controller 22, a signal receiver 30 and a signal output part31.

The image processing part 21 is, for example, a hardware circuit such asASIC and has functions to perform a variety of image processing. As apart of the functions, the image processing part 21 includes an areasetting part 23 and an image recognition part 24. The area setting part23 sets a target area that is an area intended for a process ofrecognizing an object, in an outside-vehicle image obtained by thecamera 3. The image recognition part 24 recognizes an object based onthe target area in the outside-vehicle image. In this embodiment, theimage recognition part 24 recognizes a preceding vehicle that is anothervehicle running in front of the own vehicle 1, as an object.

The controller 22 includes a microcomputer and controls the imagerecognition apparatus 2 overall. A microcomputer, for example, includesCPU, ROM, RAM and the like. A variety of functions as the controller 22are performed by CPU performing arithmetic processing in accordance witha program stored in ROM. The controller 22 includes a warning controller25, a position identifying part 26, an area changing part 27, adisplacement-amount deriving part 28 and an informing controller 29 as apart of functions performed by arithmetic processing.

The warning controller 25 takes control to warn a user to avoid acollision between the own vehicle 1 and a preceding vehicle in a casewhere the image recognition part 24 recognizes the preceding vehicle.

The position identifying part 26 identifies an optical axis position ofthe camera 3 relative to the own vehicle 1 based on an outside-vehicleimage.

The area changing part 27 changes a position of the target area in theoutside-vehicle image according to the optical axis position of thecamera 3.

The displacement-amount deriving part 28 derives an amount ofdisplacement of the optical axis position of the camera 3 from aninitial position. The initial position is the optical axis position ofthe camera 3 at the time the camera 3 is first installed in the ownvehicle 1 at a factory or a sales company of the own vehicle 1. In otherwords, the optical axis position of the camera 3, which is pointed in apredetermined direction relative to the own vehicle 1 so that the imagerecognition apparatus 2 fulfills recognition functions properly, is theinitial position.

In a case where the amount of displacement of the optical axis positionof the camera 3 from the initial position is more than a predeterminedthreshold value, the informing controller 29 takes control to inform auser that the amount of displacement of the optical axis position of thecamera 3 from the initial position is more than the predeterminedthreshold value.

The signal receiver 30 is connected with the camera 3 electrically andis an interface to receive a signal from the camera 3. The signalreceiver 30 receives an outside-vehicle image obtained by the camera 3and inputs the outside-vehicle image to the image processing part 21 andthe controller 22.

The signal output part 31 is connected with a display 4 and a speaker 5electrically and is an interface to transmit a signal to the display 4and the speaker 5. The signal output part 31 outputs the outside-vehicleimage and the like processed by the image processing part 21 to thedisplay 4 so that the display 4 displays the outside-vehicle image andthe like. In addition, the signal output part 31 transmits a signal tothe speaker 5 so that the speaker 5 outputs sound.

<1-3. Function of an Image Processing Part>

Next, functions of the image processing part 21 (the area setting part23 and the image recognition part 24) are explained in more detail.

The area setting part 23 sets a target area in an outside-vehicle image.The image processing part 21 obtains the outside-vehicle image from thecamera 3 in a predetermined cycle. The area setting part 23 sets thetarget area on respective outside-vehicle images obtained in such apredetermined cycle. The area setting part 23 sets a part of an area ofthe outside-vehicle image as the target area.

The image recognition part 24 recognizes a preceding vehicle based onsuch a target area set in the outside-vehicle image. The imagerecognition part 24 recognizes the preceding vehicle using a patternmatching method. Concretely, the image recognition part 24 extracts asubject image included in the target area, and then recognizes thesubject image as an image of the preceding vehicle in a case where ashape of the subject image approximates a standard pattern indicating avehicle.

If the image recognition part 24 recognizes a preceding vehicle bycovering the whole area of the outside-vehicle image, a processing loadfor the recognition is extremely high. Therefore, the image recognitionpart 24 recognizes the preceding vehicle by covering a target area thatis a part of the outside-vehicle image, so that the processing load isreduced.

It is preferable that the target area is determined in the positionwhere there is a high possibility that an image of the preceding vehicleappears in the outside-vehicle image. In the outside-vehicle image, animage of the preceding vehicle and a vanishing point often overlap.Therefore, the target area is set so that the target area includes thevanishing point corresponding to a theoretical infinite distance in theoutside-vehicle image. In fact, the vanishing point is a point whereimages of a plurality of lines which are parallel to one another (e.g. alane mark, etc.) are extended and intersect in the outside-vehicle imageon which a subject image is represented in perspective.

FIG. 5 shows an example of an outside-vehicle image G that is obtainedby the camera 3 and shows a forward view of the own vehicle 1. As shownin FIG. 5, a target area D is set so that the target area D includes avanishing point V in the outside-vehicle image G. In this way, apreceding vehicle is recognized by covering only the target area D setin the outside-vehicle image G, and therefore a processing load for therecognition of the preceding vehicle can be reduced.

<1-4. Process of a Controller>

Next, functions of the controller 22 (the warning controller 25, theposition identifying part 26, the area changing part 27, thedisplacement-amount deriving part 28 and the informing controller 29)are explained in more detail.

First, the warning controller 25 is explained. The warning controller 25examines a possibility of collision between the own vehicle 1 and apreceding vehicle in a case where the preceding vehicle is recognized bythe image recognition part 24. The warning controller 25 examines thepossibility of collision between the own vehicle 1 and the precedingvehicle based on a relative distance between the own vehicle 1 and thepreceding vehicle, and relative acceleration between the own vehicle 1and the preceding vehicle.

The warning controller 25 derives the relative distance based on a sizeof an image of the preceding vehicle in an outside-vehicle image. Inaddition, the warning controller 25 derives the relative accelerationbased on a change in the size of the image of the same preceding vehiclerespectively included in a plurality of consecutive outside-vehicleimages. Then the warning controller 25 examines the possibility ofcollision between the own vehicle 1 and the preceding vehicle withreference to a table stored in ROM of the controller 22 based on thederived relative distance and relative acceleration. In this table, acriterion value indicating the possibility of collision (possibility ofcontact) is stored, corresponding to the relative distance and relativeacceleration.

In addition, the warning controller 25 warns a user to avoid thecollision in a case where the criterion value indicating the possibilityof collision is higher than a predetermined threshold value. Concretely,the warning controller 25 transmits a signal to the speaker 5 so thatthe speaker 5 outputs warning sound inside a cabin of the own vehicle 1.The warning controller 25 changes volume level of warning sound outputby the speaker 5 according to the criterion value indicating thepossibility of collision.

Next, the position identifying part 26 is explained. The positionidentifying part 26 identifies an optical axis position of the camera 3relative to the own vehicle 1 based on an outside-vehicle image. In acase where the optical axis position of the camera 3 relative to the ownvehicle 1 is changed, a vanishing point position in the outside-vehicleimage is also changed. Therefore, the position identifying part 26substantively identifies the optical axis position of the camera 3relative to the own vehicle 1 by deriving the vanishing point positionin the outside-vehicle image.

When the vanishing point position in the outside-vehicle image isderived, first, an image of an object commonly included in a pluralityof consecutive outside-vehicle images is identified. Then the positionof the image of the applicable object in each of a plurality ofconsecutive images is determined. An object used in this time is, forexample, a telephone pole and the like existing along a road on which avehicle is running. The position identifying part 26 derives a pluralityof lines parallel to one another actually from the position of the imageof such an object. Then the position identifying part 26 derives theposition of a point where these plural lines intersect, as the vanishingpoint position.

For example, the outside-vehicle image G shown in FIG. 5 includes atelephone pole image X and Y along a road on which a vehicle is running.As shown in FIG. 6, the position identifying part 26 derives a pluralityof lines XL and YL parallel to one another actually based on a positionof the telephone pole image X and Y in each of a plurality ofconsecutive outside-vehicle images G. On a plurality of consecutiveoutside-vehicle images G, the telephone pole image X exists in aposition X1, X2, X3, X4 and X5 respectively. The line XL is derived byconnecting the approximate middles of these telephone pole images X.Also, in a plurality of consecutive outside-vehicle images G, thetelephone pole images Y exists in a position Y1, Y2, Y3, Y4 and Y5respectively. The line YL is derived by connecting the approximatemiddles of these telephone pole images Y. Then, a position of a pointwhere the line XL and YL intersect by extending those lines is thevanishing point position V.

In a case where a vanishing point position in an outside-vehicle imageis changed from the vanishing point position (hereinafter, referred toas a “previous position”) at the time when the area changing part 27changed the position of the target area last time, the positionidentifying part 26 derives the amount of change of the vanishing pointposition from the previous position.

If an optical axis position of the camera 3 is displaced relative to theown vehicle 1 due to aged deterioration and the like, the vanishingpoint position in the outside-vehicle image is also displaced. Forexample, as shown in FIG. 7, the vanishing point V exists inapproximately the middle in a vertical direction of an outside-vehicleimage G2. However, the vanishing point V exists above the middle in avertical direction of an outside-vehicle image G3 obtained after timehas passed, and the vanishing point V exists further above in anoutside-vehicle image G4 obtained after further time has passed.

In the case where the vanishing point position is displaced in this way,the position identifying part 26 derives the amount of change of thevanishing point position from the previous position. For example, in acase where the outside-vehicle image G3 shown in FIG. 7 is obtained, ifthe previous position is the same as the position of the vanishing pointV in the outside-vehicle image G2, the amount of change of the positionof the vanishing point V in the outside-vehicle image G3 relative to theposition of the vanishing point V in the outside-vehicle image G2 isderived. Also, in a case where the outside-vehicle image G4 shown inFIG. 7 is obtained, if the previous position is the same as the positionof the vanishing point V in the outside-vehicle image G3, the amount ofchange of the position of the vanishing point V in the outside-vehicleimage G4 relative to the position of the vanishing point V in theoutside-vehicle image G3 is derived.

Next, the area changing part 27 is explained. The area changing part 27changes a position of a target area in an outside-vehicle imageaccording to an optical axis position of the camera 3. In other words,the area changing part 27 changes the position of the target area in theoutside-vehicle image according to the vanishing point position in theoutside-vehicle image.

The area changing part 27 changes the position of the target area to beset by the area setting part 23, based on the amount of change of thevanishing point position derived by the position identifying part 26.For example, if the position of the vanishing point V in theoutside-vehicle image G3 is changed by 25 pixels to an upward direction(plus side on Y-axis) relative to the position (the previous position)of the vanishing point V in the outside-vehicle image G2 shown in FIG.7, the area changing part 27 changes the position of the target area Din the outside-vehicle image by 25 pixels to the upward direction (plusside on Y-axis). Also if the position of the vanishing point V in theoutside-vehicle image G4 is changed by 30 pixels to an upward direction(plus side on Y-axis) relative to the position (the previous position)of the vanishing point V in the outside-vehicle image G3, the areachanging part 27 changes the position of the target area D in theoutside-vehicle image by 30 pixels to an upward direction (plus side onY-axis).

In a case where an optical axis position of the camera 3 is changed to adownward direction relative to the own vehicle 1, a vanishing pointposition in an outside-vehicle image is changed to an upward directionin this way. Thus, in the case where the optical axis position of thecamera 3 is changed to a downward direction relative to the own vehicle1 (relative to a previous optical axis position), the area changing part27 changes the position of the target area in the outside-vehicle imageto an opposite direction, an upward direction.

In a case where the optical axis position of the camera 3 is changed toan upward direction relative to the own vehicle 1 (relative to aprevious optical axis position), the vanishing point position in theoutside-vehicle image is changed to the opposite direction, a downwarddirection. Therefore, in this case, the area changing part 27 changesthe position of the target area in the outside-vehicle image to adownward direction. In a case where the optical axis position of thecamera 3 is changed leftward relative to the own vehicle 1 (relative toa previous optical axis position), the area changing part 27 changes theposition of the target area in the outside-vehicle image to the oppositedirection, the right. In a case where the optical axis position of thecamera 3 is changed rightward relative to the own vehicle 1 (relative toa previous optical axis position), the area changing part 27 changes theposition of the target area in the outside-vehicle image to the oppositedirection, the left.

Next, the displacement-amount deriving part 28 is explained. Thedisplacement-amount deriving part 28 derives an amount of displacementof the optical axis position of the camera 3 from an initial position.Concretely, the displacement-amount deriving part 28 derives the amountof displacement of the vanishing point position from the vanishing pointposition (hereinafter, referred to as “reference position”) in theoutside-vehicle image in a case where the optical axis position of thecamera 3 is in the initial position. In this way, thedisplacement-amount deriving part 28 substantively derives the amount ofdisplacement of the optical axis position of the camera 3 from theinitial position by deriving the amount of displacement of the vanishingpoint position from the reference position.

The displacement-amount deriving part 28 derives the amount ofdisplacement of the vanishing point position by accumulating the amountof change of the vanishing point position derived in the past. Forexample, if the area changing part 27 has changed the position of thetarget area twice and the amount of change of the vanishing pointposition at that time was 25 pixels and 30 pixels respectively, 55pixels by accumulation of 25 pixels and 30 pixels is equal to the amountof displacement of the vanishing point position from the referenceposition. The displacement-amount deriving part 28 is allowed todirectly derive the amount of displacement of the vanishing pointposition from the reference position, based on the reference positionand the vanishing point position derived by the position identifyingpart 26.

Next, the informing controller 29 is explained. In a case where theamount of displacement of the optical axis position of the camera 3 fromthe initial position is more than a predetermined threshold value, theinforming controller 29 takes control to inform a user that the amountof displacement of the optical axis position of the camera 3 from theinitial position is more than the predetermined threshold value.Concretely, the informing controller 29 judges whether the amount ofdisplacement of the vanishing point from the reference position, derivedby the displacement-amount deriving part 28, is more than apredetermined threshold value or not. In the case where the amount ofdisplacement of the vanishing point is more than the threshold value,the informing controller 29 judges that the amount of displacement ofthe optical axis position of the camera 3 is more than the predeterminedthreshold value. In this case, the informing controller 29 outputs apredetermined signal to the display 4 and the speaker 5 so that thedisplay 4 and the speaker 5 inform a user that the optical axis positionof the camera 3 needs to be adjusted.

A threshold value is set to a value equivalent to the case where theoptical axis of the camera 3 is moved to the degree that a precedingvehicle cannot be recognized. As described previously, the area changingpart 27 changes the position of the target area according to thedisplacement of the optical axis position of the camera 3. However, in acase where one end of the target area contacts with an edge of anoutside-vehicle image and if the optical axis position of the camera 3is displaced further, it is not possible to make a further change to theposition of the target area in the direction of the edge of theoutside-vehicle image. In other words, it is not possible to recognize apreceding vehicle. Thus, the amount of displacement of the vanishingpoint equivalent to the case where one end of the target area contactswith an edge of the outside-vehicle image is set as the threshold value.It is acceptable to inform a user that the optical axis position of thecamera 3 should be adjusted in the case where one end of the target areacontacts with an edge of the outside-vehicle image.

Such a process makes it possible to inform a user that the optical axisof the camera 3 is displaced to the degree that a preceding vehiclecannot be recognized. Also a user can understand that an adjustment ofthe optical axis of the camera 3 is required.

<1-5. Flow of Process>

Next, a flow of process performed by the image recognition apparatus 2is explained.

FIG. 8 shows a flowchart of a collision-warning process that is a mainprocess to be performed by the image recognition apparatus 2. In thecollision-warning process, a preceding vehicle is recognized and awarning is given to a user in a case where there is a high possibilityof collision between the own vehicle 1 and the preceding vehicle. Thecollision-warning process shown in FIG. 8 is repeated in a predeterminedcycle (e.g. 1/30 seconds) after the image recognition apparatus 2 ispowered on and a predetermined operation is begun by a user.

FIG. 9 shows a flowchart of an optical-axis-related process that is oneof processes preformed by the image recognition apparatus 2. In theoptical-axis-related process, a response is made for the case where anoptical axis of the camera 3 is displaced from an initial position. Thisoptical-axis-related process is repeated in a relatively-long periodcycle (e.g. one month).

<1-5-1. Collision-Warning Process>

First, a flow of a collision-warning process shown in FIG. 8 isexplained. The camera 3 captures a forward view of the own vehicle 1 andobtains an outside-vehicle image showing a forward view of the ownvehicle 1. This outside-vehicle image is input to the image processingpart 21 through the signal receiver 30 (a step SA1).

Next, the area setting part 23 sets a target area in an outside-vehicleimage (a step SA2). A position of the target area to be set is stored inRAM and the like in advance.

Next, the image recognition part 24 performs a recognition process bycovering the target area in the outside-vehicle image. Thereby, theimage recognition part 24 recognizes a preceding vehicle in a case wherethe preceding vehicle exists in front of the own vehicle 1 (a step SA3).

Next, the warning controller 25 derives a relative distance between theown vehicle 1 and the preceding vehicle and relative accelerationbetween the own vehicle 1 and the preceding vehicle based on the imageof the recognized preceding vehicle. Then, the warning controller 25derives a criterion value indicating a possibility of collision betweenthe own vehicle 1 and the preceding vehicle based on the derivedrelative distance and the relative acceleration (a step SA4).

Next, the warning controller 25 judges whether the possibility ofcollision between the own vehicle 1 and the preceding vehicle is high ornot. In other words, the warning controller 25 judges whether thecriterion value indicating the possibility of collision is more than thepredetermined threshold value or not (a step SA5).

In a case where the criterion value indicating the possibility ofcollision is more than the predetermined threshold value (Yes in thestep SA5), the warning controller 25 transmits a signal to the speaker 5so that the speaker 5 outputs warning sound inside a cabin of the ownvehicle 1. Thereby, a warning is given to a user to avoid the collision(a step SA6). Such a warning makes it possible for the user to avoid thecollision between the own vehicle 1 and the preceding vehicle.

In a case where the criterion value indicating the possibility ofcollision is less than the predetermined threshold value (No in the stepSA5), the process is terminated without giving a warning.

<1-5-2. Optical-Axis-Related Process>

Next, a flow of an optical-axis-related process shown in FIG. 9 isexplained. First, the position identifying part 26 derives a vanishingpoint position in an outside-vehicle image. Thereby, the positionidentifying part 26 substantively identifies the optical axis positionof the camera 3 relative to the own vehicle 1 (a step SB1).

The position identifying part 26 derives an amount of change of thevanishing point position from a previous position in a case where thevanishing point position has changed from the previous position. In acase where a position of a target area has not been changed in the past,the vanishing point position in an outside-vehicle image (or thereference position) in a case where the optical axis position of thecamera 3 is in the initial position can be used as the previousposition.

Next, the displacement-amount deriving part 28 derives an amount ofdisplacement of the vanishing point position from the referenceposition. Concretely, the displacement-amount deriving part 28 derivesthe amount of change of the vanishing point position from the referenceposition by accumulating the amounts of change of the vanishing pointposition derived in the past and this time. Thereby, thedisplacement-amount deriving part 28 substantively derives the amount ofdisplacement of an optical axis position of the camera 3 relative to theown vehicle 1 from an initial position (a step SB2).

Next, the informing controller 29 judges whether the amount ofdisplacement of the vanishing point is more than the predeterminedthreshold value or not (a step SB3). Thereby, the informing controller29 substantively judges whether the amount of displacement of theoptical axis position of the camera 3 from the initial position is morethan the predetermined threshold value or not.

In a case where the amount of displacement of the vanishing point isless than the predetermined threshold value (No in the step SB3), inother words, in a case where the optical axis position of the camera 3is not displaced largely from the initial position, the area changingpart 27 changes a position of a target area in an outside-vehicle image.Concretely, based on the amount of change of the vanishing pointposition derived by the position identifying part 26, the area changingpart 27 changes the position of the target area to be set by the areasetting part 23 in the outside-vehicle image (a step SB4). Thereby, itis possible to recognize a preceding vehicle properly even though theoptical axis of the camera 3 is displaced from the initial position. Inthis case, a user does not have to adjust the optical axis position ofthe camera 3.

On the other hand, in a case where the amount of displacement of thevanishing point is more than the predetermined threshold value (Yes inthe step SB3), in other words, in a case where the optical axis positionof the camera 3 is displaced largely from the initial position, theinforming controller 29 informs a user that the optical axis position ofthe camera 3 is displaced largely from the initial position (a stepSB5). Concretely, the informing controller 29 outputs a predeterminedsignal to the display 4 and the speaker 5 so that the display 4 and thespeaker 5 inform that the optical axis position of the camera 3 needs tobe adjusted. Thereby, it is possible to inform the user that the opticalaxis of the camera 3 is largely displaced to the degree that a precedingvehicle cannot be recognized accurately. Thus, the user can understandthat an adjustment of the optical axis of the camera 3 is required.

As explained hereinbefore, in the image recognition apparatus 2 in thisembodiment, the image recognition part 24 recognizes a preceding vehiclebased on a target area in an outside-vehicle image obtained by thecamera 3 installed in the own vehicle 1. Then the position identifyingpart 26 identifies an optical axis position of the camera 3 relative tothe own vehicle 1 based on an outside-vehicle image, and the areachanging part 27 changes the position of the target area in theoutside-vehicle image according to the optical axis position of thecamera 3. Thereby, it is possible to recognize the preceding vehicleproperly based on the target area in the outside-vehicle image eventhough the optical axis position of the camera 3 is displaced.

<2. Second Embodiment>

Next, the second embodiment is explained. In a case where the opticalaxis position of the camera 3 is displaced largely from the initialposition, the image recognition apparatus 2 in the first embodimentinforms a user that the optical axis position of the camera 3 isdisplaced largely from the initial position. On the other hand, theimage recognition apparatus 2 in the second embodiment automaticallyadjusts the optical axis position of the camera 3 in the case where theoptical axis position of the camera 3 is displaced largely from theinitial position. A configuration and a process flow of the imagerecognition apparatus 2 in the second embodiment are approximately thesame as those in the first embodiment, and therefore, differences withthe first embodiment are explained hereinbelow.

FIG. 10 shows a configuration of the camera 3 in the second embodiment.The camera 3 in the second embodiment includes a rotation axis 17 and amotor 18 instead of the screw 14 in the first embodiment. If the motor18 is driven, the rotation axis 17 rotates. An angle relative to theholding part 12 of the chassis 11 including the optical lens 10 ischanged by the rotation of the rotation axis 17. Therefore, for thecamera 3 in this embodiment, it is possible to change a direction of theoptical axis of the camera 3 relative to the own vehicle 1 by drivingthe motor 18.

FIG. 11 mainly shows a configuration of the body part 20 of the imagerecognition apparatus 2 in the second embodiment. The controller 22 inthe second embodiment includes an optical axis adjustment part 39instead of the informing controller 29 as a function performed byarithmetic processing.

The optical axis adjustment part 39 changes a direction of an opticalaxis of the camera 3 relative to the own vehicle 1 by transmitting asignal to the motor 18 of the camera 3. In a case where an amount ofdisplacement of the optical axis position of the camera 3 from theinitial position is more than the threshold value, the optical axisadjustment part 39 adjusts the optical axis position of the camera 3.

FIG. 12 shows a flow of the optical-axis-related process in the secondembodiment. The flow of the optical-axis-related process in the secondembodiment is explained hereinbelow with reference to this figure. Theprocess of a step SC1 to SC4 shown in FIG. 12 is the same as the processof the step SB1 to SB4.

In other words, first, the position identifying part 26 derives avanishing point position in an outside-vehicle image. Also, in a casewhere the vanishing point position has changed from the previousposition, the position identifying part 26 derives an amount of changeof the vanishing point position from the previous position (a step SC1).Next, the displacement-amount deriving part 28 derives an amount ofdisplacement of the vanishing point position from the reference position(a step SC2). Then, the optical axis adjustment part 39 judges whetherthe amount of displacement of the vanishing point is more than thepredetermined threshold value or not (a step SC3).

In a case where the amount of displacement of the vanishing point isless than the predetermined threshold value (No in the step SC3), inother words, in a case where an optical axis position of the camera 3 isnot displaced largely from the initial position, the area changing part27 changes a position of a target area in an outside-vehicle image (astep SC4).

On the other hand, in a case where the amount of displacement of thevanishing point is more than the predetermined threshold value (Yes inthe step SC3), in other words, in a case where the optical axis positionof the camera 3 is displaced largely from the initial position, theoptical axis adjustment part 39 adjusts the optical axis position of thecamera 3 (a step SC5).

Concretely, the optical axis adjustment part 39 transmits a signal tothe motor 18 of the camera 3 so that the motor 18 moves the optical axisof the camera 3. In this case, based on the amount of displacement of avanishing point, the optical axis adjustment part 39 moves the opticalaxis of the camera 3 only by the amount corresponding to the amount ofdisplacement of the vanishing point. In addition, the optical axisadjustment part 39 moves the optical axis of the camera 3 in thedirection opposite to the direction where the optical axis of the camera3 is displaced relative to the own vehicle 1. For example, in a casewhere the optical axis position of the camera 3 is displaced in thedownward direction relative to the own vehicle 1, the optical axisadjustment part 39 moves the optical axis position of the camera 3 inthe opposite direction, the upward direction. Thereby, the optical axisposition of the camera 3 is returned to the initial position.

Next, the area changing part 27 also adjusts the position of the targetarea in an outside-vehicle image according to the optical axis positionof the camera 3 after adjustment (a step SC6). Concretely, the areachanging part 27 changes the position of the target area to the initialposition including the reference position of the vanishing point.Thereby, it is possible to recognize a preceding vehicle properly.

In this way, in the image recognition apparatus 2 in this embodiment,the optical axis of the camera 3 is automatically adjusted even thoughthe optical axis of the camera 3 is largely displaced to the degree thatthe preceding vehicle cannot be recognized properly. Thereby, a user canskip the step to adjust the optical axis position of the camera 3.

<3. Modification Examples>

The embodiments in the invention have been described so far. Theinvention is not limited to the above embodiments but possible to beapplied to various embodiments. Hereinbelow, modifications areexplained. Of course, each of the embodiments explained above and belowcan be arbitrarily combined with one or more of the others.

In the embodiments described above, the explanation is given as follows:a preceding vehicle C running in front of the own vehicle is recognizedas an object based on the outside-vehicle image G as shown in FIG. 13.On the other hand, a vehicle H such as a bicycle and a motorcycle isallowed to be recognized as the object. Also, a thing other than avehicle such as a pedestrian is allowed to be recognized as the object.

Not only a thing but also a pattern and the like on a surface of a thingand the like is allowed to be recognized as the object. For example, asshown in FIG. 14, a lane mark LL and RL that define the traffic lane inwhich the own vehicle 1 is running are allowed to be recognized as theobject. In this way, in a case where lane marks are recognized, it ispreferable to set the target area D below a vanishing point in theoutside-vehicle image G as shown in FIG. 14. Thereby, it is possible toset the target area D in the position where there is a high probabilityof appearance of the lane mark LL and RL in the outside-vehicle image G.

In the case where the lane marks are recognized, the warning controller25 examines, based on the lane marks, a possibility of lane departure ofthe own vehicle 1 from the traffic lane in which the own vehicle 1 isrunning, and if the lane departure is likely to be caused, it is allowedthat the warning controller 25 warns a user that there is a highpossibility of the lane departure. Concretely, the warning controller 25derives a distance between the own vehicle 1 and the respective rightand left lane marks LL and RL, and derives acceleration that the ownvehicle 1 approaches one of two lane marks LL and RL. Then, the warningcontroller 25 derives a criterion value indicating the possibility oflane departure of the own vehicle 1 from the traffic lane based on thederived values. In a case where the criterion value is higher than thepredetermined threshold value, the warning controller 25 warns a user toavoid the lane departure.

Also, in the above embodiments, the explanation is given as follows: theangle of view of the optical lens 10 of the camera 3 is 45 degrees. Onthe other hand, as the optical lens of the camera 3, it is allowed toadopt a wide-angle lens whose angle of view is relatively wide. It ispreferable that the angle of view of such a wide-angle lens is more than100 degrees, further preferable that the angle of view is more than 120degrees, much further preferable that the angle of view is more than 180degrees.

FIG. 15 shows an example of the outside-vehicle image G obtained by thecamera 3 with a wide-angle lens. As shown in FIG. 15, in a case wherethe wide-angle lens is adopted, the outside-vehicle image G includes thewide area spreading right and left in front of the own vehicle 1 as asubject. In this case, as shown in FIG. 15, it is allowed to set thetarget areas D at the right side and the left side of theoutside-vehicle image G respectively. As described above, it is possibleto recognize an object existing on the right side or left side of theown vehicle 1, which easily becomes a blind area for a driver, bysetting the target areas D on both sides of the right and the left sideof the outside-vehicle image G. Thereby, a user can recognize an objectsuch as a vehicle and a pedestrian approaching from a blind area whenthe own vehicle 1 enters an intersection with bad visibility, and thelike. As a result, it is possible to avoid a collision with such avehicle or a pedestrian.

Also, in the above embodiments, the explanation is given as follows: thecamera 3 is installed on the inner surface of the windshield 15 and theoptical axis of the camera 3 points ahead of the own vehicle 1. On theother hand, it is allowed to place the camera 3 on a sideview mirror ofthe own vehicle 1 so that the optical axis of the camera 3 points in theleft-hand direction or the right-hand direction of the own vehicle 1.

Thus, in a case where the optical axis of the camera 3 is pointed in thelateral direction of the own vehicle 1, it is preferable to adopt awide-angle lens as the optical lens of the camera 3. FIG. 16 shows anexample of the outside-vehicle image G obtained in a case where theoptical axis of the camera 3 with a wide-angle lens is pointed in theright-hand direction of the own vehicle 1. In this case, as shown inFIG. 16, the outside-vehicle image G includes the area from the front toback on the right side of the own vehicle 1 as a subject. Also in thiscase, as shown in FIG. 16, it is preferable to set the target area D onthe right and left side of the outside-vehicle image G respectively.Thereby, it is possible to recognize a vehicle, a pedestrian and thelike approaching from a forward and backward area that easily become ablind area, as an object. Thereby, a user can recognize a vehiclerunning in the adjacent traffic lane and approaching the own vehicle 1,a pedestrian walking close to the own vehicle 1 and the like. As aresult, it is possible to avoid a collision with such a vehicle and apedestrian.

In this case where the optical axis of the camera 3 is pointed in thelateral direction of the own vehicle 1, it is difficult to figure outthe vanishing point position in the outside-vehicle image G. Thus, it isallowed that the optical axis position of the camera 3 relative to theown vehicle 1 is substantively identified by recognizing the lane mark Ldefining the traffic lane in which the own vehicle 1 is running based onthe outside-vehicle image G and by deriving the position of the lanemark L in the outside-vehicle image G.

It is allowed that the camera 3 is installed on the rear window of theown vehicle 1 and the optical axis of the camera 3 is pointed in thebackward direction of the own vehicle 1. In this case, it is possible torecognize a vehicle approaching from the backward direction of the ownvehicle 1 and a pedestrian existing behind the own vehicle 1, as anobject.

Also, in the above embodiments, in a case where there is a highpossibility of collision, a warning of high possibility of collision isgiven to a user to avoid the collision between the own vehicle 1 and anobject. On the other hand, in the case where there is a high possibilityof collision, it is allowed that the controller 22 transmits apredetermined signal to a vehicle controller that controls the behaviorof the own vehicle 1 so that this vehicle controller controls the engineand the brake of the own vehicle 1 in order to avoid the collisionbetween the own vehicle 1 and the object.

While the invention has been shown and described in detail, theforegoing description is in all aspects illustrative and notrestrictive. It is therefore understood that numerous othermodifications and variations can be devised without departing from thescope of the invention.

What is claimed is:
 1. An image recognition apparatus that recognizes anobject based on an image, the image recognition apparatus comprising: arecognition unit that recognizes the object based on a target area in animage obtained by a camera installed in a vehicle; an identifying unitthat, during normal operation of the vehicle, identifies an optical axisposition of the camera relative to the vehicle based on the image; and achanging unit that, during normal operation of the vehicle, changes aposition of the target area in the image according to the optical axisposition identified by the identifying unit.
 2. The image recognitionapparatus according to claim 1, further comprising: a derivation unitthat derives an amount of displacement of the optical axis position froman initial position based on the image; and an informing unit thatinforms a user that the amount of displacement is more than a thresholdvalue in a case where the amount of displacement is more than thethreshold value, wherein the changing unit changes the position of thetarget area in the image in a case where the amount of displacement isless than the threshold value.
 3. The image recognition apparatusaccording to claim 1, further comprising: a derivation unit that derivesan amount of displacement of the optical axis position from an initialposition based on the image; and an adjustment unit that adjusts theoptical axis position in a case where the amount of displacement is morethan a threshold value, wherein the changing unit changes the positionof the target area in the image in the case where the amount ofdisplacement is less than the threshold value.
 4. The image recognitionapparatus according to claim 1, wherein the changing unit changes theposition of the target area in the image in a first direction oppositeto a second direction, in a case where the optical axis position haschanged in the second direction relative to a previous optical axisposition.
 5. The image recognition apparatus according to claim 1,wherein the identifying unit identifies the optical axis position byderiving a vanishing point position in the image.
 6. The imagerecognition apparatus according to claim 1, wherein the objectrecognized by the recognition unit is another vehicle that is running infront of the vehicle in which the camera is installed.
 7. The imagerecognition apparatus according to claim 6, wherein the target areaincludes a vanishing point in the image.
 8. The image recognitionapparatus according to claim 1, wherein the object recognized by therecognition unit is lane marks that define a traffic lane in which thevehicle runs.
 9. The image recognition apparatus according to claim 3,wherein the adjustment device comprises a motor.
 10. The imagerecognition apparatus according to claim 5, wherein the identifying unitderives the vanishing point from an object commonly included in aplurality of consecutive outside-vehicle images.
 11. An imagerecognition method for recognizing an object based on an image, themethod comprising the steps of: (a) recognizing the object based on atarget area in an image obtained by a camera installed in a vehicle; (b)during normal operation of the vehicle, identifying an optical axisposition of the camera relative to the vehicle based on the image; and(c) during normal operation of the vehicle, changing a position of thetarget area in the image according to the optical axis positionidentified in step (b).
 12. The image recognition method according toclaim 11, further comprising the steps of: (d) deriving an amount ofdisplacement of the optical axis position from an initial position basedon the image; and (e) informing a user that the amount of displacementis more than a threshold value in a case where the amount ofdisplacement is more than the threshold value, wherein the step (c)changes the position of the target area in the image in a case where theamount of displacement is less than the threshold value.
 13. The imagerecognition method according to claim 11, further comprising the stepsof: (d) deriving an amount of displacement of the optical axis positionfrom an initial position based on the image; and (e) adjusting theoptical axis position in a case where the amount of displacement is morethan a threshold value, wherein the step (c) changes the position of thetarget area in the image in a case where the amount of displacement isless than the threshold value.
 14. The image recognition methodaccording to claim 11, wherein in the step (c), the position of thetarget area in the image is changed in a first direction opposite to asecond direction, in a case where the optical axis position has changedin the second direction relative to a previous optical axis position.15. The image recognition method according to claim 11, wherein in thestep (b), the optical axis position is identified by deriving avanishing point position in the image.
 16. The image recognition methodaccording to claim 11, wherein in the step (a), the object that isrecognized is another vehicle that is running in front of the vehicle inwhich the camera is installed.
 17. The image recognition methodaccording to claim 16, wherein the target area includes a vanishingpoint in the image.
 18. The image recognition method according to claim11, wherein in the step (a), the object that is recognized is lane marksthat define a traffic lane in which the vehicle runs.
 19. The imagerecognition method according to claim 13, wherein the optical axisposition is adjusted by a motor.
 20. The image recognition methodaccording to claim 15, wherein the vanishing point is derived from anobject commonly included in a plurality of consecutive outside-vehicleimages.